1. Field of the Invention
The present invention relates to the cooling of enclosures, such as racks, for diverse types of equipment, such as heat-producing electronics, through a combination of air and liquid cooling for very high total power levels of the equipment.
For instance, as heat is generated during operation of electronic equipment, such as that comprising an integrated-circuit chip (IC), the thermal resistance between chip junction and the medium employed for the removal of heat must be sufficiently small in order to provide a junction temperature that is low enough to insure the continued reliable operation of the equipment. However, the problem of adequate heat removal becomes ever more difficult to solve as chip geometry is scaled down and operating speeds of the electronic equipment are increased, resulting in an increased power density (W/cm2) at the surface of the chip. The problem is further exacerbated when different types of chips in close proximity with each other possess different cooling requirements. For example, in a computer system, a processor chip may have a much higher power density than closely located memory chips. Furthermore, as another example, different types of chips may have different maximum-allowable junction temperatures. Such cooling requirements impose mechanical and thermal packaging challenges to the equipment design that can limit the performance thereof. In the current technology, the power density of processor and other kinds of high-performance chips is rapidly approaching levels that exceed the capability of forced-air cooling, necessitating the use of liquid cooling for some applications and installations in order to be able to attain the requisite degree of cooling for the equipment.
The cooling of computer racks and other types of electronic equipment is typically accomplished by forced-air cooling; however, liquid-assisted air cooling and direct-liquid cooling, frequently with water as the cooling medium, have also been widely employed. This concept is discussed in Richard C. Chu, et al., “Review of Cooling Technologies for Computer Products”, IEEE Transactions on Device and Materials Reliability, Vol. 4, No. 4, pp. 568-585, (December 2004). In liquid-assisted air cooling, a liquid-cooled heat exchanger is placed in a heated air stream in order to extract heat and reduce the air temperature before it is expelled into the room. Chu, et al. (Supra) also describe the problems encountered with data-center thermal management, in which the power dissipated for each equipment rack is approaching 30 kW. In a typical modern data center, water-cooled air-conditioning units or other external cooling devices are used to provide, through perforations in a raised floor or through ducts, a stream of chilled air to the computers, in which the air is heated, and downstream of which the air is returned to air-conditioning units so as to be chilled again. Significant problems encountered with this approach include the need for the large circulatory volume of air required to adequately cool the electronics, the extensive raised-floor space required to handle this air volume, the accompanying high acoustic noise levels encountered in the room, and the difficulty of controlling the airflow in the room to prevent already-hot air from re-circulating into the electronics, thereby potentially leading to overheating and electronic failure of the equipment. Moreover, the computer machine room can be uncomfortable for human occupancy because of large temperature differences present between room areas cooled by the cold inlet air and room areas heated by the hot outlet air. It is noted hereby that traditional data-center cooling is basically quite similar to liquid-assisted air cooling in that, in both instances, heat is initially transferred from the electronics to air. The difference resides in the location of the subsequent heat transfer from air to liquid: in traditional data-center cooling, this air-to-liquid heat transfer occurs outside the computer racks, typically in air-conditioning units where the liquid is water, whereas in liquid-assisted air cooling, it occurs within the computer racks.
2. Discussion of the Prior Art
Various methods and apparatus have been developed in the technology for the purpose of imparting adequate cooling to diverse types of operating equipment, such as electronic devices functioning at high power levels and which generate considerable amounts of heat, which must be dissipated.
Chu, et al., U.S. Pat. No. 6,819,563 B1, which is commonly assigned to the present assignee, and the disclosure of which is incorporated herein by reference, discloses a method and system for augmenting the air cooling of rack-mounted electronics systems by using a cooling fluid to cool air entering the system, and to remove a portion of the heat dissipated by the electronics being cooled. A drawback in the adding of heat exchangers to an electronic rack is due to an increased flow resistance that reduces airflow through the rack. In the patent, the air path is an open loop, whereas the present invention is directed to the provision of a closed-loop air path inside the rack.
Chu, et al., U.S. Pat. No. 6,775,137 B2, which is commonly assigned to the present assignee, and the disclosure of which is incorporated herein by reference, relates to an enclosure apparatus that provides for a combined air and liquid cooling of rack-mounted, stacked electronic components. A heat exchanger is mounted on the side of the stacked electronic components, and air flows from the front to the back within the enclosure, impelled by air-moving devices mounted behind the electronic components. A drawback in adding a heat exchanger to the side of an electronics rack is the requirement for an increase in floor space. Moreover, a front-to-back airflow within the confines of the rack does not allow for the use of a continuous midplane for the electronic components.
Patel, et al, U.S. Pat. No. 6,628,520 describes an apparatus for housing electronic components that includes an enclosure, mounting boards with electronic components mounted thereon, a supply plenum for cooling air, one or more outlets, which are directed toward the mounting boards, one or more heat-exchanging devices, and one or more blowers. A significant limitation in this arrangement resides in that inlet and outlet plenums for the air are needed along opposite sides of the electronics rack, in addition to the space required at the top and bottom of the rack, which is used to reverse the direction of the air flow.
Ishimine, et al., U.S. Pat. No. 6,621,707 pertains to an electronic apparatus comprising a motherboard, multi-chip modules mounted to the motherboard, cooling members for cooling the multi-chip modules, a refrigeration unit for cooling the cooling members to room temperature or lower, and a connection structure to releasably couple each multi-chip module thermally and mechanically to the refrigeration unit. In contrast therewith, the present invention does not use a refrigeration unit, or require a substantially hermetically scaled box structure, or a drying means for supplying dry air for cooling purposes.
Sharp, et al, U.S. Pat. Nos. 6,506,111 B2 and 6,652,373 B2 each describe a rack with a closed-loop airflow and a heat exchanger. The air flows vertically up one side of the rack, horizontally across electronics devices, vertically down the other side of the rack, and then across a heat exchanger located in the base, or optionally on the tops, of the rack. Because the air path is much shorter for memory cards near the heat exchanger location, a perforated plate is included in one of the vertical paths to enable adjustment of the airflow across the various memory cards to match some desired, e.g., constant distribution. The present invention does not require the vertical plenums, which occupy valuable space, or the perforated plate.
Parish IV, et al., U.S. Pat. No. 6,462,949 B1 discloses a cooling apparatus using “low-profile extrusions” to cool electronic components mounted on a board or card, whereas contrastingly, the present invention does not use any “low-profile extrusions” or similar structure.
Miller, et al., U.S. Pat. No. 6,305,180 B1 discloses a system for cooling electronic equipment using a chiller unit between adjacent racks for returning cooled air to ambient. Contrastingly, the present invention is distinct from the foregoing because in the system described therein, the air is re-circulated within the rack rather than being expelled to the ambient environment.
Go, et al., U.S. Pat. No. 5,144,531 is directed to a liquid cooling system comprising cold plates attached to their respective circuit modules, quick couplers for connecting flexible hoses to these cold plates, a supply duct, and a return duct to form strings of cold plates, which are connected between the supply duct and the return duct. Valved quick couplers are used for the connection to the supply duct and the return duct, and valveless quick couplers are used for the connection to the cold plates. In contrast, pursuant to the present invention, a quick connect is not used to connect to the cold plate, though quick connects may be employed for the connections to each individual blade.
Koltuniak, et al., U.S. Pat. No. 3,749,981 describes a modular power supply wherein the power modules, each with its own fans, are mounted inside a sealed cabinet. Also mounted inside the cabinet are cooling modules, each with its own fan and heat exchanger. This patent represents an early example in the technology of an air-recirculation system requiring shared airflow plenums that occupy valuable space.
Ward, et al., U.S. Pat. No. 3,387,648 pertains to a cabinet-enclosed cooling system for electronic modules mounted on a modular chassis, wherein the chassis is extensible from the cabinet. This is an example of an air-recirculation system that requires, at the front and back of the assembly, shared vertical air plenums which unnecessarily occupy valuable space.
In implementing the construction of high-performance computer systems, it is desirable to be able to electrically interconnect as many processor chips and memory cards as possible while using conventional and economically priced electronic packaging methods. Thereby, the more densely and closely packed the electronics are, the more difficult they are to cool, because space is required for air circulation and for heat sinks. One method of achieving dense packaging of the electronic components is to build modular units called “blades”, each of which contains one or more processors and memory card(s). Multiple blades are then plugged into a common electrical backplane, or midplane, which, because of its high wiring density, provides for a high-speed and cost-effective inter-blade communication. Moreover, the modularity of blades allows for the sharing of common system resources, and facilitates servicing and configuration changes. Blade-type packaging is not limited to computer systems, but may also be employed for switch systems, or other types of information processing, and for matching and/or mixing of different functions within a single rack or enclosure.
Two features of conventional blade-style packaging essentially limit the performance achievable by the electronic components located within a rack:
1. Front-to-Back Airflow
Racks with blade-style packaging frequently employ vertical backplanes (or midplanes) in conjunction with front-to-back airflow cooling arrangements, thereby requiring airflow holes to be formed in the backplane. Such holes, to a significant extent, block wiring channels in the backplane, thereby greatly reducing the number of I/O's (input/output electrical signaling interconnections) available for connection to the attached blades. Moreover, in such a rack, the relatively small airflow cross-section provided by the holes in the backplane limits total power dissipation to about 30 kW. This aspect is disclosed in the publication by M. J. Crippen, et al., “BladeCenter packaging, power, and cooling”, IBM J. Res. & Dev., Vol. 49 No. 6, Nov. 2005, pp. 887-904.
2. Total Reliance on Air-Cooling
As a cooling fluid, air is advantageous vis-a-vis water because it effortlessly bathes myriad heat-producing electronic devices in a safe, insulating cooling fluid. However, air is disadvantageous in comparison with water because its small heat capacity per unit volume, 3500 times smaller than water, limits the power density that may be cooled, and requires a considerable amount of airflow space, which restricts packaging density.
The above-mentioned features of conventional blade-style packaging, front-to-back airflow and total reliance on air cooling, must be clearly improved upon in order to solve the following problems, which are currently in evidence:
(a) limited total power that can be dissipated in a blade-style rack,
(b) limited packaging density due to space required for airflow,
(b) high engineering cost of customized airflow solutions for conventional raised-floor data centers,
(c) excessive data-center noise encountered due to air movers and airflow, and
(d) discomfort encountered by personnel in data centers due to non-uniform air temperatures.